Predictive coding is an important technique in video coding. It is motivated by the fact that adjacent pixels in video signals often have similar values, thus it is more efficient to encode the difference between the pixels. In predictive coding, a sample is not encoded directly. Rather, an input sample is first predicted from a few previously reconstructed samples. Then the prediction error (residue) is quantized and entropy coded. The reconstructed value at the decoder is the predicted value plus the quantized residue. To guarantee the encoder and decoder use exactly the same prediction value, the encoder must use the same process as the decoder to reproduce reconstructed samples. This is called closed-loop prediction.
The International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) Standard/International Telecommunication Union, Telecommunication Sector (ITU-T) H.264 Recommendation (hereinafter the “MPEG-4 AVC Standard”) is a video coding standard which employs spatial directional prediction for intra coding. When using intra prediction, pixels in a current block are predicted from its causal neighboring blocks. In addition to applying prediction in the spatial domain, the encoder can predict a block from previous frames through motion estimation and compensation. Such temporal prediction is very effective when the underlying content is stationary.
In the MPEG-4 AVC Standard, spatial intra prediction is formed using surrounding available samples, which are previously reconstructed samples available at both the encoder and decoder within the same slice. When the picture is encoded at a low bit rate, the reconstructed samples are distorted from the original samples due to large quantization errors. Since the previously reconstructed samples are used for intra prediction in the current block, the prediction errors become larger as the bitrate gets lower and the predictive coding is less efficient.
Spatial directional prediction for intra coding performed in accordance with the MPEG-4 AVC Standard provides a more flexible prediction framework, thus the coding efficiency is better than when compared to previous standards in which intra prediction was performed in the transform domain. As previously described, in the MPEG-4 AVC Standard, a spatial intra prediction is formed using surrounding available samples, which are previously reconstructed samples available at both the encoder and decoder within the same slice. For luma samples, intra prediction can be done on a 4×4 block basis (denoted as Intra—4×4), an 8×8 block basis (denoted as Intra—8×8) and on a 16×16 macroblock basis (denoted as Intra—16×16). Turning to FIG. 1, MPEG-4 AVC Standard directional intra prediction with respect to a 4×4 block basis (Intra—4×4) is indicated generally by the reference numeral 100. Prediction directions are generally indicated by the reference numeral 110, image blocks are generally indicated by the reference numeral 120, and a current block is indicated by the reference numeral 130. In addition to luma prediction, a separate chroma prediction is performed. There are a total of nine prediction modes for Intra—4×4 and Intra—8×8, four modes for Intra—16×16 and four modes for the chroma component. The encoder typically selects the prediction mode that minimizes the difference between the prediction and original block to be coded. A further intra coding mode, I_PCM, allows the encoder to simply bypass the prediction and transform coding processes. It allows the encoder to precisely represent the values of the samples and place an absolute limit on the number of bits that may be contained in a coded macroblock without constraining decoded image quality.
Turning to FIG. 2, labeling of prediction samples for the Intra—4×4 mode of the MPEG-4 AVC Standard is indicated generally by the reference numeral 200. FIG. 2 shows the samples (in capital letters A-M) above and to the left of the current blocks which have been previously coded and reconstructed and are therefore available at the encoder and decoder to form the prediction.
Turning to FIGS. 3B-J, Intra—4×4 luma prediction modes of the MPEG-4 AVC Standard are indicated generally by the reference numeral 300. The samples a, b, c, . . . , p of the prediction block are calculated based on the samples A-M using the Intra—4×4 luma prediction modes 300. The arrows in FIGS. 3B-J indicate the direction of prediction for each of the Intra—4×4 modes 300. The Intra—4×4 luma prediction modes 300 include modes 0-8, with mode 0 (FIG. 3B, indicated by reference numeral 310) corresponding to a vertical prediction mode, mode 1 (FIG. 3C, indicated by reference numeral 311) corresponding to a horizontal prediction mode, mode 2 (FIG. 3D, indicated by reference numeral 312) corresponding to a DC mode, mode 3 (FIG. 3E, indicated by reference numeral 313) corresponding to a diagonal down-left mode, mode 4 (FIG. 3F, indicated by reference numeral 314) corresponding to a diagonal down-right mode, mode 5 (FIG. 3G, indicated by reference numeral 315) corresponding to a vertical-right mode, mode 6 (FIG. 3H, indicated by reference numeral 316) corresponding to a horizontal-down mode, mode 7 (FIG. 31, indicated by reference numeral 317) corresponding to a vertical-left mode, and mode 8 (FIG. 3J, indicated by reference numeral 318) corresponding to a horizontal-up mode. FIG. 3A shows the general prediction directions 330 corresponding to each of the Intra—4×4 modes 300.
In modes 3-8, the predicted samples are formed from a weighted average of the prediction samples A-M. Intra—8×8 uses basically the same concepts as the 4×4 predictions, but with a block size 8×8 and with low-pass filtering of the predictors to improve prediction performance.
Turning to FIGS. 4A-D, four Intra—16×16 modes corresponding to the MPEG-4 AVC Standard are indicated generally by the reference numeral 400. The four Intra—16×16 modes 400 includes modes 0-3, with mode 0 (FIG. 4A, indicated by reference numeral 411) corresponding to a vertical prediction mode, mode 1 (FIG. 4B, indicated by reference numeral 412) corresponding to a horizontal prediction mode, mode 2 (FIG. 4C, indicated by reference numeral 413) corresponding to a DC prediction mode, and mode 3 (FIG. 4D, indicated by reference numeral 414) corresponding to a plane prediction mode. Each 8×8 chroma component of an intra coded macroblock is predicted from previously encoded chroma samples above and/or to the left and both chroma components use the same prediction mode. The four prediction modes are very similar to the Intra—16×16, except that the numbering of the modes is different. The modes are DC (mode 0), horizontal (mode 1), vertical (mode 2) and plane (mode 3).
During the development of the ITU-T H.26L Standard, displaced intra prediction was proposed. The proposal re-uses the concept of variable block size inter-prediction as specified in the MPEG-4 AVC Standard for intra prediction.
Turning to FIG. 5A, an example of displaced intra prediction is indicated generally by the reference numeral 550. The displaced intra prediction 550 involves an intra coded region 552, a current block 554, and a candidate block 556. In general, previously encoded intra regions (e.g., intra coded region 552) of a slice can be referenced by displacement vectors (e.g., displacement vector 556) for prediction of the current intra block (e.g., current block 554). The displaced intra prediction 550 is implemented on a macroblock basis. The displacement vectors are encoded differentially using a prediction by the median of the neighboring blocks, in analogy to the inter motion vectors in the MPEG-4 AVC Standard.
Template matching prediction (TMP) is a concept of texture synthesis to deal with the generation of a continuous texture that resembles a given sample. Intra prediction using template matching in the context of the MPEG-4 AVC Standard has been proposed. In the proposal, the scheme is integrated as an additional mode for Intra—4×4 or Intra—8×8 prediction in the MPEG-4 AVC Standard. With template matching prediction, self-similarities of image regions are exploited for prediction. Previously encoded intra regions of a slice can be reused for prediction. The TMP algorithm recursively determines the value of current pixels under prediction by selecting at least one patch (of one or more pixels) of decoded data. Patches are selected according to a matching rule, where patch neighboring pixels are compared to current block neighboring pixels, and patches having the most similar neighboring pixels are selected. Turning to FIG. 5B, an example of template matching intra prediction is indicated generally by the reference numeral 570. The template matching intra prediction 570 involves a candidate neighborhood 572, a candidate patch 574, a template 576, and a target 578. Since the search region and the neighborhood (e.g., candidate neighborhood 572) of the current pixels (e.g., target 578) are known at the encoder and the decoder side, no additional side information has to be transmitted, and identical prediction is achieved on both sides.
Both DIP and TMP have improved the coding efficiency, at the cost of high computational complexity. Similar to other MPEG-4 AVC Standard spatial intra prediction modes, the prediction in DIP and TMP is formed using surrounding available samples. When the picture is encoded at a low bit rate, the reconstructed samples are distorted from the original samples due to large quantization errors, the prediction errors become larger, and the predictive coding is less efficient.